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Deflning and Ensuring Academic Rigor in Online and On-Campus Courses: Instructor 

Perspectives 



Charles Graham 
Christopher Essex 
Indiana University 



Abstract 



This study attempted to define what academic rigor means to the faculty members and graduate assistants 
who taught on -campus and online courses at a major Midwestern state university. The study sought to 
discover the level of importance these instructors gave to academic rigor and what strategies they used to 
ensure academic rigor in their courses. Finally, the researchers asked instructors about their strategies to 
ensure academic rigor in these two different delivery methods. 

Introduction 

Academic rigor is a topic that resides at the very core of the traditional conception of the academy. While 
precise definitions (Winston et al., 1994; Braxton, 1993; Nicholson, 1996) may vary, the basic idea is central to 
postsecondary education as it has been implemented in the past few hundred years. However, at the dawn of this 
new millennium, the popularity of new pedagogical beliefs and instructional strategies, such as constructivism and 
problem-based learning, and delivery methods, such as online distance education, make it clear that it is time to 
review our conceptions of academic learning and to see if old definitions match the postsecondary educational 
practices of the new millennium. 

The amazing growth of online distance education, in particularly, heightens the need for this type of study. 
As enabling technologies such as computers and the Internet have become more ubiquitous in the U.S., distance 
education programs have become increasingly popular in both formal academic settings as well as corporate 
settings. Specifically, over the past five years there has been a move by many institutions of higher education to 
provide some kind of online learning opportunities. According to a recent MSNBC report (McGinn, 2000), 75 
percent of all U.S. universities now offer online coursework, and 5.8 million students have taken online college 
courses. This rapid movement towards offering online courses has caused some concern, especially in academic 
communities, regarding the wisdom and implications of this trend. One of the concerns that is often voiced (Phipps 
& Merisotis, 1999; Rutmania, 1999) implies that online courses are not as academically rigorous as traditional face- 
to-face courses. If this is truly the case then the academic reputation of the institution could possibly be at risk. 

This study attempted to explicitly define what academic rigor means to the faculty members and graduate 
assistants who taught on-campus and online courses at a major Midwestern state university. The study also sought to 
discover the level of importance these instructois gave to academic rigor as they designed their course offerings, and 
what strategies they used to ensure academic rigor in their courses. Finally, the researchers asked those instructors 
who taught both online and on-campus courses about their strategies to ensure academic rigor in these two different 
delivery methods. 

Research Questions 

This research study attempts to answer three primary questions: 

How do instructors define academic rigor? 

How important is academic rigor to these instructors? 

What strategies do these instructors use to ensure academic rigor in their on-campus courses? 

For those instructors who teach both on -campus and online, what strategies do these instructors use to 
ensure academic rigor in their online courses? Are these strategies different than the strategies used in their on- 
campus courses? 

Our hypotheses going into the research were that instructors would have a variety of definitions of 
academic rigor, but that important similarities would emerge, and that the differences in definitions would also be of 
interest. We assumed most instructors would view academic rigor to be of great importance in their course design. 
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We thought that instructors would have a variety of strategies for ensuring academic rigor and that, again, we would 
find a core group of strategies, with interesting variations to report as well. Our hypothesis regarding the difference 

ensuring rigor in online and on-campus courses, based on informal conversations with instructors of both 
types of courses prior to the research, would be minimal, and that similar strategies, though obviously influenced by 
the medium used to deliver the course, would be employed. 

Significance of Research 

There are many reasons why this research is significant to instructors and administrators at postsecondary 
institutions, especially those institutions embarking upon distance education initiatives. Three key areas of impact 
are described below. 

Evaluation and improvement of current on-campus and online courses 

Most instmctors, we feel it is fair to say, would agree that there is a need for the continual evaluation and 
improvement of their course offerings. If our hypothesis that academic rigor is important to instructors is correct, 
then our findings relating to the first research question should provide some clear criteria regarding academic rigor 
which then can be used to evaluate the extent to which it exists in current course offerings. Additionally, the findings 
of the third and fourth research questions will most likely shed light on specific strategies relating to academic rigor 
in on-campus and online courses that can be focused on to improve the academic rigor of both types of courses. 

Promoting discussion of academic rigor among faculty 

We hope that the results of the research will help to promote discussion of academic rigor and related 
pedagogical issues among faculty. By reading about others’ strategies for ensuring academic rigor, they may be 
moved to discuss these strategies and their own with their colleagues. 

Providing an opportunity for distance educators to share their beliefs about, and strategies to ensure, 
academic rigor 

Any study that looks at the concept of academic rigor and does not also discuss it in the context of online 
distance education is doing a disservice to its readers, given the prevalence of these courses today (McGinn, 2000). 
Distance educators have been accused of providing less-than-rigorous courses voiced (Phipps & Merisotis, 1999; 
Rutmania, 1999) and we wish to give them an opportunity to dispel these notions, if they are indeed incorrect in the 
eyes of the instructors involved. 

Past Research 

Our review of the relevant research shows that the topic of academic rigor is mentioned throughout the 
literature, but that only in a few instances is the meaning of the term specified. We were only able to find three 
cases in which the term academic rigor was explicitly defined in some way. Below we include the three different 
definitions. Being that the concept is a complicated one, rather than trying to restate and summarize the elaborate 
definitions we encountered, and risk losing some of the authors’ meaning, we will present below three of the best 
definitions that we found in their entirety. 

An environment that is intellectually challenging and demanding. Students perceive a norm of excellence 
and responsibility, which is expressed through high, but realistic, evaluation standards. The class is seen as fast- 
paced, and there are expectations that students will invest considerable energy and time in completing assignments 
(Winston et al., 1994) 

Academic quality is manifested in such course-level academic processes as the type of questions faculty ask students 
during class, the nature of term paper assignments or other written exercises. (...) [AR is] the level of 
understanding of course content to be demonstrated by students while engaging in these course-level processes. 
(Braxton, 1993) 

Rigor — focused and critical work — arises from a sense of the importance of subject matter and the 
opportunity presented for its mastery and refinement through study. (...) In a rigorous academic environment, the 
purposes, principles and methodologies of scholarship as a means of establishing the connectedness of things is 
understood. (Nicholson, 1996) 
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All three of these definitions have different foci. While Winston’s definition focuses on student output and 
perception of a course, Braxton’s definition focuses on the instructor’s efforts as well as the learning outcomes of 
the course and Nicholson’s definition centers around a more traditional awareness and use of methodologies. 

Although most of the articles that referred to academic rigor did not explicitly define the term they often 
had embedded within the text phrases that have helped us to gain a better feel for what they meant by academic 
rigor. Table 1 has a list of key phrases relating to academic rigor from some of the articles that were reviewed. It is 
important to note that despite the fact that academic rigor is used in many different ways, it is almost always referred 
to as being a positive attribute that a program or course should have under ideal circumstances. 



Reference Terms 



Ridley et al., 1998 


“course achievement as measured by grades” 

“concern that instructors have applied comparable standards in assigning grades to their 
online or traditional students” 


Snell etal., 1999 


“Online or distance learning is harder than offline or traditional classroom learning” 
“it appeared that online courses were more rigorous” 

“less withdrawals and failures among the traditional offline group” 


Thomas, 1998 


“academically challenging” 

“number of homework assignments” 


Kerr, 1990 


“Depth of study in areas such as foreign languages and mathematics” 

“Quality: Four or five academic subjects each year” 

“Balance: Evidence that the student took a broad curriculum” 

“Trends: Evidence as to whether the student’s grades are gradually improving each year” 


Rossman et al., 1996 


“increased requirements” 

“more rigorous content standards” 

“students maintain a specific grade point average” 
“more challenging program of study” 

“taking more courses” 

“taking more challenging courses” 

“taking more course breadth” 


Taylor et al., 1991 


“critical thinking” 

“mastery of a body of facts” 

“development of writing skills” 

“acquisition of library and research skills” 

“substantial reading and writing assignments” 

“analysis and interpretations of primary sources” 

“assignments that foster critical thinking and interpretative skills” 


Niles et al., 1990 


“liberal education” 

“equip individuals with the skills and understanding necessary to perform their duties” 


Bursuck, 1994 


“educational rigor” 

“higher standards” 

“increased accountability” 

“higher expectations for student performance” 
“assigning more homework to students” 


Roundtable, 1997 


“academic rigor demanded by the national curriculum” - content coverage? 
Skills vs. knowledge outcomes in curriculum 


Hayes, 1997 


“the ultimate test of the student’s academic preparation and intellectual ability . . . comes 
at the time of university admission exams” 

“academic standards” 

“academic demands” 

“scholarship and academic excellence” 

“academic achievement” 

“academic effort” 



Table 1. Key phrases related to academic rigor from the literature. 



Craig Nelson, a Biology professor at Indiana University, spoke at Indiana University’s 1 8*^ Annual Spring 
Symposium, Listening to Learners: Creating Contexts for Student Success, The title of his presentation was “How 
We Defeat Ourselves: Dysfunctional Illusions of Rigor.” Although his talk did not directly address what academic 
rigor is, it shed light on the subject by addressing what he felt academic rigor is not. The illusions of rigor that he 
presented expressed the idea that rigor is not: 

• Hard courses that weed out weak students 

• Avoiding “pampering” students (e .g., giving them flexibility etc.) 

• Covering more content (Nelson, 1989) 

• The difficulty of the exams 

(Nelson, 2000a) 

In an article (1997) entitled “Tools for Tampering with Teaching’s Taboos,” Nelson promotes the 
importance of “critical thinking” in the college classroom. He provides several methods or tools for getting students 
to think critically about what they are learning. 

Methodology 

The goal of the research was to answer the research questions outlined above. The following general steps 
were taken in this effort. 

The first step of this phase was to determine how other individuals and institutions have defined academic 
rigor, by conducting an exhaustive review of the past research in this area. Because of the variations of academic 
rigor, and the fact that the concept was often referred to under different names, this research effort was quite 
challenging. 

The findings from the literature review were used to inform the questions that were asked in an initial set of 
interviews of a purposefully chosen group of instructors at the aforementioned large Midwestern state university. 
These instructors were chosen from throughout the university’s many schools and departments (see Table 2). The 
purpose of this initial data gathering was to acquire a larger body of information regarding academic rigor from 
which more targeted and precise questions could be created for the final survey. 

In addition to the demographic information presented in Table 2, it is important to note that of the 
instructors interviewed for this study (n=8), 2 were full professors, 3 were associate, visiting or adjunct faculty, and 
3 were advanced graduate students serving as associate instructors. Of this group of instructors, 3 had taught online 
as well as on-campus, and thus our findings for question 4 will be based on responses from these instructors. 



Department 


Gender 


Years Teaching Online 


Years Teaching On -campus 


School of Nursing 


Female 


2 


12 


Leisure and Recreation 


Male 


0 


3 


Biology 


Male 


0 


34 


English & School of 
Continuing Studies 


Female 


0 


3 


Telecommunications 


Female 


0 


5 


Language Education 


Male 


2 


5 


Curriculum and 
Instruction 


Male 


0 


25 


Instructional Systems 
Technology 


Female 


3 


5 



Table 2, Demographic information about interview participant 

The questions posed of our interview subjects were as follows: 

• What is your personal definition of academic rigor? 

• How important do you feel academic rigor is? Why? 

• What do you do to ensure academic rigor in your courses? 

• Do you do different things in your online courses compared with your on-campus courses to 
ensure academic rigor? If so, what? 
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The final step in our research study procedure was to review and summarize the data collected through the 
interviews. Similarities and differences in the perceptions related to academic rigor were noted. 



Findings 



Definition of Academic Rigor 



What it IS 


What W IS NOT 


critical thinking 


not grades 


high standards and expectations 


not memorization 


process more than product 


not regurgitation 


cognitive development 





Table 3. Common responses to the academic rigor definition question 



What it IS 


Wliat it /.S' NOT 


norm referenced 

content coverage 

gatekeeper of academy 

scholarship/peer review 

student involvement 

pushing students beyond comfort level 


not time based 

not giving lots of tests 

not lots of people who fail 



Table 4. Interesting but uncommon responses to the academic rigor definition question. 

Our analysis of the interview data related to the definition of academic rigor developed into a two separate 
listings: of common responses to what academic rigor was and was not (Table 3), and interesting but less common 
responses to the question (Table 4). Given that the responses to the question had such interesting variations, we felt 
it was necessary to report them, especially given that survey data might potentially find that the alternate definitions 
might be held by a significant number of respondents. 

The Importance of Academic Rigor 

Based on our interviews with faculty members, we found: 

• As expected, all participants agreed that academic rigor is important 

• Two instructors said that its importance varied depending on the level of the course (e.g., rigor 
was more important for higher-level courses) 

Methods for Ensuring Academic Rigor with On-campus Courses 

After analyzing the data from our interviews with faculty members, we found a diverse collection of 
strategies that nonetheless seemed to fit well together. We divided these strategies into seven categories, with one or 
more strategy associated with each category. Unfortunately, it is beyond the scope of this study to go into specific 
details about how each strategy should be implemented, but hopefully this list of strategies should be of benefit to 
online educators. 

Expectations: 

• Make expectations clear 

• Make expectations high 

• Make grading criteria explicit 
Selection of Readings: 

• Provide high quality readings/texts 

• Provide materials a step above students’ level 

• Provide a variety of materials 
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Critical Thinking: 

• Require and support students* efforts to examine multiple issues from multiple perspectives 

• Require students to cite the readings 

• Require student reflections 

Modeling: 

• Model good scholarship 

• Model rigorous thinking 

Support: 

• Provide appropriate scaffolding for learners 

Discourse: 

• Ask questions that encourage thinking about relationships and not memorization 

Assignments: 

• Create challenging assignments 

• Create assignments that require higher cognitive processes 

• Design authentic and realistic projects and problems for the students 

Methods for Ensuring Academic Rigor with Online Courses 

Based on our interview data, we found that, in general, similar methods for ensuring academic rigor seem 
to be used for online as well as on-campus courses, which means that the list directly above should be relevant also 
for online courses. However, some instructors did note some differences in their teaching methods related to 
academic rigor in the online delivery method. Some instructors felt that, in their online courses, they: 

• Put more demands on students because of the absence of feedback fro m face-to-face contact 

• Structured online discussions more carefully 

• Made expectations even more explicit than they did with on-campus students 

• Expended more effort to get feedback from students 

Interestingly, one instructor went on to say that if content coverage was equated to academic rigor then: 

• The same methods will work online as in the classroom 

• The online course can be more academically rigorous than the on -campus course because of up-to- 
date resources 

However, he stated that, in his mind, rigor was not the same as content coverage, and that other things, such 
as the quality of instructor-student and student -student interaction, were as important, if not more so, than merely 
providing specified content, and thus required different strategies for online delivery. He did not, unfortunately, 
specify these differing strategies. 

Rigor, Difficulty, and Support 

One of the faculty interviewees, Craig Nelson, has been nationally recognized for excellence in teaching at 
a post-secondary level. Nelson (2000b) provided some insights related to academic rigor that we feel are 
particularly relevant to the online delivery of courses. 

His first observation was that oftentimes faculty and students have a false conception of what academic 
rigor is. We often confuse the rigor of a course with the difficulty of a course. He claimed that a more accurate 
synonym to describe academic rigor would be the word ""challenge"" It was insightful for us to think of academic 
rigor or academic challenge as having the following relationship with course difficulty and the support provided to 
the distance students. Nelson (2000b) related these three factors using the equation in Figure 1 . 




Academic rigor 



Support 



Figure 1. Relationship between course difficulty, academic rigor and student support. 

This helps us to understand that the difficulty of a course is inversely proportional to the support that 
students are given. Thus the difficulty of a course increases as the support for the students in the course decreases. 
Similarly, the difficulty of a course could be low and the course could still be academically rigorous if adequate and 
appropriate support is provided to the students. 

This understanding of the relationship difficulty^ academic rigor, dindi support is particularly 

important in a distance learning environment because of high dropout rates. Students are more likely to drop out of 
a distance program as the courses become more difficult and exceed the students’ ability or willingness to persevere. 
Instructors teaching distance education courses, Nelson (2000b) argues, should maximize the challenge that a 
student experiences but not the difficulty that a student experiences trying to meet that challenge. 

Limitations 

It must be stated at this point, that, even though our findings from our interviews were quite interesting and 
potentially quite useful, they are based on quite a small sample size (especially research question 4, which was 
answerable by less than half of our already small sample, who taught online as well as face-to-face). Another 
important limitation is that the sample only included faculty from a single institution. 

Conclusions 

The purpose of this study was to gain a better understanding of academic rigor in online and on-campus 
postsecondary education. In order to do this accurately, the researchers attempted to define factors that instructors 
understood ’'academic rigor" to consist of, as well as quantify how important academic rigor is to these instructors. 
Information about strategies used to ensure academic rigor in online and on-campus courses was also gathered. 

While the results reported in this report are based on a very small sample size, the findings are promising, 
and the researchers feel that this study will be of great interest to both on -campus and online educators and 
administrators interested in ensuring the academic rigor of their courses. 
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